Skip to content

Fix(presto)!: preserve SHA256/SHA512 digest semantics, render SHA2 as hex string [CLAUDE]#7824

Merged
georgesittas merged 1 commit into
tobymao:mainfrom
Pawansingh3889:presto-sha2-digest-semantics
Jul 7, 2026
Merged

Fix(presto)!: preserve SHA256/SHA512 digest semantics, render SHA2 as hex string [CLAUDE]#7824
georgesittas merged 1 commit into
tobymao:mainfrom
Pawansingh3889:presto-sha2-digest-semantics

Conversation

@Pawansingh3889

Copy link
Copy Markdown
Contributor

Presto and Trino's native SHA256/SHA512 take and return VARBINARY, but the parser mapped them to exp.SHA2, the string-to-hex-string expression that MySQL and Spark use. The codebase already models this split with exp.MD5 and exp.MD5Digest, and the presto parser maps native MD5 to MD5Digest one line above the SHA mappings, so this looked like an oversight rather than a decision.

Two changes, both mirroring the MD5 precedent:

  • The parser now maps native SHA256/SHA512 to exp.SHA2Digest. Digests survive a round trip and transpile to other digest functions: Trino SHA256(x) becomes UNHEX(SHA256(x)) in DuckDB and stays SHA256(x) in BigQuery, instead of silently turning into a hex-string function.
  • The generator renders exp.SHA2 the way md5_sql renders exp.MD5: LOWER(TO_HEX(SHA256(TO_UTF8(x)))), with the UTF8 encode applied when the argument is typed as text. Spark's SHA2(x, 256) now produces the same value on Trino that it produces on Spark. SHA224/SHA384 raise the usual unsupported warning since Presto has no equivalents.

Updated the bigquery and exasol expectations that encoded the old equivalence. Exasol's HASH_SHA256 returns a hex string, so LOWER(TO_HEX(SHA256(x))) is the value-preserving translation there; its presto/trino read entries asserted that a digest function and a hex-string function were the same thing, which is the bug this fixes, so those two entries are removed.

Found while chasing a surrogate key mismatch in SQLMesh on Trino (SQLMesh/sqlmesh#5871): MD5-based keys transpile correctly, SHA256-based keys silently change value.

One adjacent inconsistency deliberately left alone: the bigquery parser maps SHA256 to SHA2Digest but SHA512 to SHA2, while BigQuery returns BYTES for both. Happy to follow up separately if that is wanted.

@geooo109 geooo109 self-assigned this Jul 6, 2026

@georgesittas georgesittas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Left a few comments.

Comment on lines +511 to +515
if not this.type:
from sqlglot.optimizer.annotate_types import annotate_types

this = annotate_types(this, dialect=self.dialect)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this. For transpilation purposes, nowadays we assume that type inference has ran separately and thus we expect is_type on its own to suffice.

Just need to make sure the Presto functions roundtrip unaffected by default, i.e., when there aren't any type annotations in the AST. I think this is already the case, just double-checking.

Comment on lines +1176 to 1178
"presto": "LOWER(TO_HEX(SHA512(x)))",
"trino": "LOWER(TO_HEX(SHA512(x)))",
},

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a test where you annotate types in the ast you get after parsing with bigquery, so that the transpilation path with is_type is reached in a realistic scenario.

Comment on lines +738 to +753
# string-semantics SHA2 renders hex output like md5_sql does for MD5
self.validate_all(
"SELECT SHA2(x, 256)",
write={
"presto": "SELECT LOWER(TO_HEX(SHA256(x)))",
"trino": "SELECT LOWER(TO_HEX(SHA256(x)))",
},
)
self.validate_all(
"SELECT SHA2(x, 512)",
write={
"presto": "SELECT LOWER(TO_HEX(SHA512(x)))",
"trino": "SELECT LOWER(TO_HEX(SHA512(x)))",
},
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These aren't valid presto inputs afaict, i.e., SHA2(x, 256) and SHA2(x, 512). We should use read={...} and use a dialect that supports these calls to demonstrate actual transpilation so that the test is well-formed.

@geooo109 geooo109 removed their assignment Jul 6, 2026
@georgesittas

Copy link
Copy Markdown
Collaborator

I'll take this to the finish line, thanks.

@georgesittas georgesittas merged commit 802b7ec into tobymao:main Jul 7, 2026
8 checks passed
@Pawansingh3889 Pawansingh3889 deleted the presto-sha2-digest-semantics branch July 7, 2026 13:51
@Pawansingh3889

Copy link
Copy Markdown
Contributor Author

Thanks for taking it over the line. Both review points noted for next time, and the bigquery SHA512 follow-up offer stands if it's wanted.

@georgesittas

Copy link
Copy Markdown
Collaborator

No problem! You can see the amended commit where I applied my suggestions here.

@Pawansingh3889

Copy link
Copy Markdown
Contributor Author

On the bigquery SHA512 follow-up: it ripples further than the SHA256 line. Clickhouse needs SHA2Digest wiring, and the new annotated-types test would change expectation, since BigQuery's SHA512 returns BYTES and the value-preserving presto form is SHA512(TO_UTF8(x)) rather than the hex string. That also suggests sha2_digest_sql wants the same typed-text encode handling as sha2_sql. Happy to do all of that as one PR if the direction sounds right.

@georgesittas

Copy link
Copy Markdown
Collaborator

Go for it and we can chat about the details on the PR :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants